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The purpose of this self-study guide is to help adult literacy education providers collect, organize, and 
analyze evidence that they can use to improve program performance. It was designed to help educators 
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important for evaluating implementation. Sources of evidence for this review include records and data 
such as lesson plans, rosters, and student results confirming that processes are in place to monitor teacher 
and student success. The components important to evaluation of implementation were determined 
based on a thorough review of the literature on adult education. The guide was developed in partnership 
with the Regional Educational Laboratory Southeast’s Florida Career Readiness Research Alliance. It was 
pilot tested with Florida adult literacy educators through the support of the Institute for the Professional 
Development of Adult Educators. 
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INTRODUCTION 


Literacy programs in adult basic education, with limited resources, aim to address the needs of a large popula- 
tion in the United States. Although the relationship between adult literacy and well-being has been documented 
(DeWalt, Berkman, Sheridan, Lohr, & Pignone, 2004; Nutbeam 2008), educators are often unsure how to imple- 
ment literacy programs (Miller, Esposito, & McCardle, 2011). The purpose of this guide is to help adult literacy 
education providers collect, organize, and analyze evidence they can use to improve program performance. 


This self-study guide was developed in partnership with the Regional Educational Laboratory (REL) Southeast’s 
Florida Career Readiness Research Alliance. Feedback from alliance members and program administrators in their 
districts helped shape the guide. The guide was pilot tested with Florida adult literacy educators through the 
support of the Institute for the Professional Development of Adult Educators. 


The guide was developed to benefit states, districts, colleges, and programs implementing adult literacy edu- 
cation. It was designed to help educators consider which types of evidence to collect and which components of 
adult education instruction may be important for evaluating implementation. Sources of evidence for this review 
include records and data such as lesson plans, rosters, and student results confirming that processes are in place 
to monitor teacher and student success. The components important to evaluation of implementation were deter- 
mined based on a thorough review of the literature on adult education; they are outlined below. 


Purpose and use of the self-study guide 


The purpose of the Self-study Guide for Evidence-Based Practices in Adult Literacy Education is to help states, 

districts, colleges, institutions, schools, and programs: 

e Collect information for developing an implementation plan for adult literacy education. A list of possible sources 
of evidence to consider on the status of current program implementation is provided for each component 
area. Sources of evidence may then be identified and discussed as the Planning and Implementation Scoring 
Guide is completed (see below). 

e Prioritize needs for improvement while developing the implementation plan for adult literacy education. Individ- 
ual and group ratings allow colleagues to discuss and prioritize components for improvement. 

° Collect data on program and student performance in different areas of implementation—referred to as 
progress-monitoring information—for continuous program improvement. 

e Evaluate the implementation of adult literacy education through discussion with stakeholders and the develop- 
ment of an action plan. 


The guide aims to promote reflection on current strengths and challenges in planning or implementation, initiate 
conversations among staff, and identify areas for improvement. Teachers and others who deliver adult literacy 
education, along with advisors and program-based administrators, should participate in the self-study process. 


Adult literacy educators who pilot tested this guide determined that the beginning or end of the school year was 
the best time to conduct a self-study. This allows educators to reflect on implementation in the previous year and 
plan for the next year. 


The self-study process takes approximately three to five hours. Time estimates are provided in the steps outlined 
in box 1. It is helpful to elicit input from participating teachers and others who deliver adult literacy education, in 
addition to advisors and program-based administrators. 


The self-study process works best when led by a dedicated facilitator. The facilitator should be a careful listener 
and able to lead and structure discussions around collected evidence and decisionmaking processes. The facili- 
tator should be knowledgeable in instructional policies, procedures, and implementation of adult literacy educa- 
tion practices and research. The facilitator should review the guide before the process begins, which should take 
approximately two hours. 


Components of the guide 


This guide has five components: 

e Planning and Implementation Scoring Guide. 

e Possible Sources of Evidence for Consideration. 

e Implementation Majority Agreement Rating Form. 
e Planning Next Steps Form. 

e Action Plan Template. 


Planning and Implementation Scoring Guide 


The Planning and Implementation Scoring Guide includes guiding questions to support planning and implemen- 
tation of adult literacy education programs. The guiding questions are included to stimulate the thinking of team 
members on practices that may be considered for each area. It will be most helpful for facilitators to select team 
members who are knowledgeable about these practices in schools and districts so that team members will have 
information on how these practices are implemented locally. 


The Planning and Implementation Scoring Guide’s six areas were identified by the REL Southeast after a review 
of the research and practice literature (see appendix A). That review suggested that the following areas could be 
important for educators to consider when planning and implementing adult literacy education programs: 


¢ Development and use of student data profiles. 
e Instructional methods and materials. 

e Instructional time. 

e Teacher preparation and training. 

¢ Communication. 

e Instructional environment and processes. 


Box 1 lists the steps for completing the Planning and Implementation Scoring Guide. 
Possible Sources of Evidence for Consideration 


Before convening a meeting of the self-study team, the facilitator should consider sources of evidence and create 
a one-page list of the types of evidence. The Possible Sources of Evidence for Consideration section of the guide 
may be helpful for this purpose. The list of the types of evidence collected by the facilitator will be used with the 
discussion that occurs during the group rating process (described in Step 5 in box 1). 


Implementation Majority Agreement Rating Form 


After all self-study team members have completed the Planning and Implementation Scoring Guide, the facilitator 
should guide the team through a group rating process. The team should use the Implementation Majority Agree- 
ment Rating Form to reach agreement on the current status of implementation of the adult literacy education 
program and to plan the next steps. The most important part of this process is the discussion that goes into group 
rating. Box 1 lists the steps for completing the Implementation Majority Agreement Rating Form. 


Box 1. Steps to complete the Planning and Implementation Scoring Guide, Implementation Majority 
Agreement Rating Form, and Planning Next Steps Form 


1. Recruit 5-7 members for the self-study team, and convene a meeting to complete the self-study process. Select a 
dedicated and knowledgeable facilitator. Then recruit teachers, others who deliver adult literacy education, volunteers, staff 
members, facilitators, and relevant program-based administrators (program managers, site managers, department chairs, 
coordinators, curriculum specialists, and advisors) knowledgeable in adult literacy education policies and implementation to 
complete the team. 


2. Present an overview of the self-study process to the team, including a review of data and possible sources of evi- 
dence identified by the facilitator. [Activity length: 30 minutes] 


3. Ask each team member to review the content of the Planning and Implementation Scoring Guide for each area to be 
rated (for example, Development and use of student data profiles, Instructional methods and materials, and Instruc- 
tional time) and the Support for Planning and Implementation Scoring Guide areas (appendix A). [Activity length: 
20 minutes] 


4. Answer team members’ questions about the review. [Activity length: 20 minutes] 


5. Have each team member complete a numerical rating for each area in the Planning and Implementation Scoring 
Guide. Ratings (1-4) should be based on team members’ existing knowledge and discussion of evidence sources identified by 
the facilitator. Each team member should rate each area independently to provide individual feedback. Team members may 
abstain from rating areas that they do not feel competent to rate. [Activity length: 60 minutes] 


6. Rate asa group to reach majority agreement. There are several steps to group rating [Activity length: 90 minutes]: 


e Rate. Ask each team member to provide the numerical rating selected for each of the six areas. 

e Discuss the rationale for high frequency numbers. Ask someone who selected a high frequency number to talk about what 
motivated that rating. 

e Discuss the rationale for lower frequency numbers. Ask other members to talk about what motivated their ratings. 

e Rate. Ask each member to provide a second numerical rating. Members may change their initial ratings based on the 
discussion. 

¢ Record ratings. \f there is majority agreement (determined by majority vote), record the high frequency number on the 
Implementation Majority Agreement Rating Form. If agreement is not reached (there is no high frequency number), contin- 
ue discussing and rating until majority agreement is reached. 

Repeat this process for each area until majority agreement is reached for each area. 


7. Discuss and record initial team thoughts on priorities, next steps, and activities on the Implementation Majority 
Agreement Rating Form. [Activity length: 20 minutes] 


8. Complete the Planning Next Steps Form. The facilitator leads a discussion with the team about the priorities for action and 
then about development of a detailed implementation plan for next steps and activities that are most urgent and actionable. 
Finally, the facilitator leads a discussion to identify potential challenges to the plan and uses that information to complete the 
Action Plan Template. [Activity length: 60 minutes] 


Planning Next Steps Form and Action Plan Template 


The Planning Next Steps Form is used to prioritize the elements in the six areas based on the evidence shared 
during majority agreement discussions and on each area’s importance to the success of the adult literacy edu- 
cation program. The self-study team should review the group rating scores together. After this review, the team 
should highlight areas revealing a need for development or improvement and identify two or three top priorities 
from the six areas for action planning. The team should then record the priority areas and complete a detailed 
plan for next steps and activities, noting any potential challenges. Once this step is complete, the facilitator should 
use the information to complete the Action Plan Template. 


PLANNING AND IMPLEMENTATION 
SCORING GUIDE 


Area 1: Development and use of student data profiles 


“Assessing a student’s reading ability is important in programs where reading instruction is individualized or 
where growth in reading is monitored” (p. 35). “An important finding in the Adult Education research suggests 
that assessing several components of reading in order to generate profiles of students’ reading ability gives teach- 
ers much more instructionally relevant information than any test of a single component can” (p. 35). 


Kruidenier, J. R., MacArthur, C. A., & Wrigley, H. S. (2010). Adult education literacy instruction: A review of the research. 
Washington, DC: National Institute for Literacy. 


Part 1.1 A data collection process, beginning at student intake, is in place 
to collect and share data on student assessment and planning for proper 
program and instructional placement 


Guiding questions 


To help you determine a numerical rating for this area, consider each question and then answer Yes (Y), No (N), or 
Developing (D). 


Does the program’s master schedule indicate who will deliver adult literacy education, during what times, 
in what location, for which students, and in what size group? 


Are there clear eligibility requirements for determining who will receive adult literacy education? 

Does the school determine what level of support eligible students will receive through instruction? 

Does the program use teacher recommendations, previous academic experience, writing samples, inter- 
views, or existing assessment data to assess students’ adult literacy education needs and place students in 
the correct level of instruction as soon as possible after a student enters the program? 

Are students asked if they have been previously diagnosed with a disability? 


Are students asked if they have had previous formal schooling in the United States or elsewhere? 


Are students’ goals for the program, postsecondary training, and career identified to assist with 
placement? 


Does someone ensure that all students’ literacy needs have been identified so students are placed appro- 
priately and so that teachers are aware of those needs? 


Is there a more efficient way to identify students’ literacy needs? 


_____ Does someone interpret the results of the academic assessments and translate the findings into 
instruction? 


_____ Is there a process for developing a clear postsecondary or career articulation plan for students in the 
program aligned to student literacy proficiency and goals? 


Part 1.2 Teachers/programs are able to track student progress through the 
use of valid and reliable assessments 


Guiding questions 


To help you determine a numerical rating for this area, consider each question and then answer Yes (Y), No (N), or 
Developing (D). 


Do plans used to track student progress ensure consistency in data collection, aggregation, analysis, and 
review? 


Are assessments administered to monitor academic progress? 


Are data analyzed for instructional use and to assign students to sections or small groups according to 
instructional need? 


Does the program have a thorough process to identify and select valid and reliable standardized literacy 
assessments? 


Does the program ensure that multiple assessments are not redundant? 


Does evidence show that the assessments can predict literacy success in postsecondary education and the 
workplace? 


Does the program align progress monitoring assessments, student goals, teacher curriculum frameworks, 
required standards, and outcome measures? 


Does the program offer regular and frequent (such as biweekly) meetings on student progress? 
Can students track their own progress to see their performance growth? 


Does each student’s schedule consider progress steps and program completion requirements and focus on 
assessments targeted to student needs? 


Does the program train teachers on administering assessments, making instructional and grouping deci- 
sions, reviewing student progress, and determining changes in placement? 


Do teachers and students use data to set goals, adjust instructional practices, and guide the selection of 
curriculum materials to enhance student-centered learning, strengthen student motivation, and increase 


student engagement? 


Does the program address student retention? 


Does the program provide for student accommodations for assessments, if needed? 


Area 2: Instructional methods and materials 


“Federal and state policy makers should move quickly to build on and expand the existing infrastructure of adult 
literacy education to support the use of instructional approaches, curricula, materials, tools, and assessments of 
learners consistent with (a) research on reading, writing, learning, language, and adult development; (b) research 
on the effectiveness of instructional approaches; and (c) knowledge of sound assessment practices” (p. 252). 


Lesgold, A. M., & Welch-Ross, M. (2012). Improving adult literacy: Options for practice and research. Washington, DC: Com- 
mittee on Learning Sciences: Foundations and Applications to Adolescent and Adult Literacy, National Research Council, 
National Academy of Sciences. 


Part 2.1A plan is in place to use criteria for selecting interventions and 
curricula that have been shown to have a positive effect on student 
achievement 


Guiding questions 


To help you determine a numerical rating for this area, consider each question and then answer Yes (Y), No (N), or 
Developing (D). 


Does the program have criteria for evaluating and selecting interventions and curricular materials for use in 
classes (for example, a rubric or checklist)? 


Are all components of selected interventions or curricula available in their entirety to ensure that each 
intervention or curriculum is delivered as intended (with fidelity)? 


Are all students able to engage in personalized learning (to work and practice independently at their own 
level) and to ensure that the materials align closely with student needs? 


Is there a clear process for selecting and implementing differentiated interventions for students struggling 
with early literacy development/foundational reading skills? 


Does the program provide access to instruction for students with disabilities? 


Part 2.2 A plan is in place that reflects instructional practices shown through 
empirical evidence to increase student achievement 


Guiding questions 


To help you determine a numerical rating for this area, consider each question and then answer Yes (Y), No (N), or 
Developing (D). 


Are there procedures and policies for using selected materials? 


Have individuals who will be delivering instruction received professional development to support effective 
use of selected reading interventions and curricula? 


Does someone facilitate the development of instructional (lesson) plans? 
Are lesson plans informed by student assessment data and aligned to individual student needs and skills? 


Are classroom observation data used to ensure that teachers are implementing best practices during 
instruction? 


Do classroom observations reveal that students are engaged in learning, and is instruction relevant and 
rigorous? 


Do observations help verify the fidelity of implementation and support for effective instructional practices 
(for example, observations of adherence to intervention components delivered with quality, teacher-created 
portfolios that include artifacts of instruction)? 

Does the program incorporate student writing into the curriculum? 


Does the program integrate literacy instruction across subject matter areas? 


Does the program differentiate instruction according to student needs in each subject area? 


Does the program use a Response to Intervention process to ensure that students with disabilities, and 
other students in need of more intensive instruction, receive the level of support needed for success? 


Area 3: Instructional time 


“One primary reason for limited progress may be that adults lack sufficient amounts of instruction and practice 
for improving skills” (p. 77). “Reasons reported for dropping out of adult education include... the pace of instruc- 
tion (either too fast or two slow) ... and inconvenient class location or schedule” (p. 77). 


Lesgold, A. M., & Welch-Ross, M. (2012). Improving adult literacy: Options for practice and research. Washington, DC: 
Committee on Learning Sciences: Foundations and Applications to Adolescent and Adult Literacy, National Research 
Council, National Academy of Sciences. 


Part 3.1 The school has established a schedule that will provide the 
appropriate frequency, consistency, and duration of instruction to meet 
adult learners’ needs 


Guiding questions 


To help you determine a numerical rating for this area, consider each question and then answer Yes (Y), No (N), or 
Developing (D). 


Does the program schedule provide adequate time (enough days per week and minutes per day) for 
instruction? 


Does the schedule devote enough time for students with disabilities and others with significant literacy 
deficits to provide the intensity of instruction needed to enable academic growth? 


Do diagnostic assessment data indicate that instructional time is adequate to meet student needs? 
Does the program offer literacy courses in a range of time slots that are convenient for adult learners? 
Is there a process to learn about schedules that are most convenient for adult learners? 

Does the program consistently implement the instructional schedule as designed? 

Do learning management systems allow students time for additional practice? 


Has the program considered competency-based education, personalized learning, or blended and online 
approaches to provide as many opportunities for students as possible? 


Has the program considered whether to co-enroll adult learners into both literacy courses and career 
readiness, postsecondary, or industry certification courses? 


Are there guidelines or policies for class sizes? 


Has the program identified subject matter experts who can provide guidance on pacing and instructional time? 
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Area 4: Teacher preparation and training 


Researchers note the importance of ensuring that adult literacy teachers are well trained and prepared for instruc- 
tion and that they are offered ongoing professional development and coaching. Examples of evidence-based 
recommendations for teacher preparation include hiring experienced, full-time teachers (p. 110) and providing 
teachers with classroom aides (p. 97). 


Kruidenier, J. R., MacArthur, C. A., & Wrigley, H. S. (2010). Adult education literacy instruction: A review of the research. 
Washington, DC: National Institute for Literacy. 


Part 4.1 A plan is in place to identify, hire, and retain faculty able to teach 
academic literacy skills effectively to adult learners 


Guiding questions 


To help you determine a numerical rating for this area, consider each question and then answer Yes (Y), No (N), or 
Developing (D). 


Does the program effectively recruit new teachers and advertise teaching positions? 


Does the program appropriately vet teacher candidates for hiring (including their instructional background 
and expertise)? 


Are there criteria to ensure that newly hired teachers can effectively instruct adult learners in academic 
literacy skills? 


Is the program taking steps to retain high-quality faculty? 
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Part 4.2 A plan is in place to provide professional development and ongoing 
support for teachers providing adult literacy education 


Guiding questions 


To help you determine a numerical rating for this area, consider each question and then answer Yes (Y), No (N), or 
Developing (D). 


_____ Do teachers receive orientation and initial training? 
Are follow-up and other professional development opportunities offered? 
Does someone ensure that teachers receive ongoing training? 


Does the program offer ongoing professional development that relates to the instructional materials used 
in the classroom? 


Are professional development offerings tied to program goals and outcomes, evidence-based practices, 
standards, and curriculum needs? 


Does the program schedule professional development offerings to meet teachers’ varying availability 
(nights, weekends)? 


Are there incentives for teachers to participate in professional development? 


Do staff receive training on how to address the needs of students beyond instruction, such as transporta- 
tion, homelessness, and health issues? 


Do staff know whom to contact for social services or other wraparound services that adult learners and 
their families might need? 


Has the program planned or implemented professional learning communities for collaborative professional 
development among staff? 


Does the program identify and select mentor teachers to support new teachers or teachers needing 
support—through observations, coaching, co-teaching, or something else? 


Does the program have a timeline that teachers and administrators have agreed on for teachers’ imple- 
mentation of instructional practices modeled during professional development? 


Does the school use rubrics or checklists to conduct ongoing observations of the fidelity of teachers’ 
instruction? 


Do classroom observations inform administrators about the kinds of support and feedback teachers may 
need? 


Area 5: Communication 


Researchers note the importance of communication and community building in ensuring that students receive 
assistance in transitioning to postsecondary training and careers. “Adult learners can face individual, institutional, 
and policy-related barriers to successful transitioning to postsecondary education. State directors of adult learner 
programs have discussed providing additional support services as a way to address these barriers” (p. 8). 


Hector-Mason, A., Narlock, J., Muhisani, H., & Bhatt, M. P. (2017). State strategies to facilitate adult learners’ transitions 
to postsecondary opportunities (REL 2017-223). Washington, DC: U.S. Department of Education, Institute of Education 
Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Midwest. 


Part 5.1 A plan is in place for communication and collaboration that will 
ensure successful startup and ongoing implementation of instructional 
programs 


Guiding questions 


To help you determine a numerical rating for this area, consider each question and then answer Yes (Y), No (N), or 
Developing (D). 


Is someone responsible for organizing and coordinating administrators, program coordinators, counselors, 
and teachers for the startup of adult literacy education programs? 


Is someone responsible for identifying school personnel and community volunteers, if used, who will 
provide instruction and support? 


Is someone responsible for identifying training opportunities for individuals interested in providing 
instruction? 


Is someone responsible for creating instructional schedules and intake procedures, ensuring timely assess- 
ment of students to determine eligibility for adult literacy education, and identifying students for instruc- 
tional placement? 


Has the program reached out to educators who have served previous cohorts of students to determine 
which prior practices were viewed as being successful? 


Is someone responsible for marketing and publicizing the program to ensure that the community under- 
stands the mission statement and services offered? 


Is someone responsible for confirming that the program connects with other agencies and service provid- 
ers to ensure that wraparound services are available for students who may need them? 
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Part 5.2 A plan is in place that enables teachers, administrators, counselors, 
and others involved in delivering adult literacy education to collaborate 
regularly on students’ growth and program success and improvement 


Guiding questions 


To help you determine a numerical rating for this area, consider each question and then answer Yes (Y), No (N), or 
Developing (D). 


Do all teachers understand the instructional goals and progress for each student and discuss student prog- 
ress with appropriate team members? 


Does the program deliver successful outcomes for students as determined by reviews of standardized 
literacy outcome measures? 


Do all teachers know how to track and share data on student progress during conferences in a way that 
students and other team members clearly understand? 


Are teachers using information gained during instruction to inform students and the adult education 
program team about progress? 


Do teachers collect student work and other informal data during instruction to share at team meetings and 
student progress conferences? 


Is information from program coordinators and counselors shared and used by teachers? 


Does someone facilitate collaborative discussions among teachers, program coordinators, and counselors 
about students’ instructional needs? 


Do students and family members receive resources to support and build students’ academic skills at home? 


Does someone ensure that students make a positive connection with program members and other stu- 
dents in the first three weeks of instruction to foster a sense of community and boost retention? 


Does someone contact students who consistently miss class? 
Does someone ensure that students have access to job fairs and postsecondary training opportunities? 


Are former students invited back to share their success stories and challenges to assist adult educators with 
program improvement? 


Is a suggestion box available for students and staff to encourage the sharing of ideas to improve the program? 
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Area 6: Instructional environment and processes 


Researchers note the importance of making the instructional environment conducive to learning for adult edu- 
cation students and of having processes in place to support student engagement and productivity. Examples of 
evidence-based recommendations for instructional environment include learner-centered classes (p. 100), com- 
munity building (p. 125), and differentiated teaching environments (p. 125). 


Kruidenier, J. R., MacArthur, C. A., & Wrigley, H. S. (2010). Adult education literacy instruction: A review of the research. 
Washington, DC: National Institute for Literacy. 


Part 6.1 A plan is in place to ensure a healthy and safe learning environment 
that is welcoming and conducive to learning 


Guiding questions 


To help you determine a numerical rating for this area, consider each question and then answer Yes (Y), No (N), or 
Developing (D). 


Does the program have criteria for creating an instructional environment that provides healthy and safe 
conditions that are conducive to learning and that minimize distractions (attractive design, print rich, 
appropriate physical space and furniture, good lighting)? 


Is the entrance to the instructional setting inviting? 


Is there a plan to regularly monitor the instructional environment to ensure that it remains a healthy and 
safe learning space throughout the school year? 


Is space consistently available to provide instruction? 

Are instructional materials and supplies readily available? 

Does the program have clear policies and procedures for customer service and intake? 
Do staff follow established policies and procedures for customer service and intake? 
Do administrators create and share assessment schedules? 

Do staff follow established assessment schedules? 


Does the program differentiate policies and processes for adult basic education, GED, and English learner 
instruction? 


Do faculty and staff create a positive and appealing instructional environment that encourages adult learn- 
ers to remain in the program and meet their goals? 


Part 6.2 The learning environment is adaptive to the social, emotional, and 
cultural needs of adult learners 


Guiding questions 


To help you determine a numerical rating for this area, consider each question and then answer Yes (Y), No (N), or 
Developing (D). 


___ Have faculty and staff received training in the social, emotional, and cultural needs of students, and are 
they aware of what services and supports are available for students? 


___ Are procedures in place for faculty and staff to identify and report concerns about social, emotional, and 
cultural needs among students so that the needs can be addressed quickly? 


Do administrators consider students’ social, emotional, and cultural needs to help students have a success- 
ful adult literacy education experience? 
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POSSIBLE SOURCES OF EVIDENCE 
FOR CONSIDERATION 


This list of possible sources of evidence for each Planning and Implementation Scoring Guide area provides facil- 
itators and self-study team members with information to review when thinking about and discussing current 
implementation of practices in adult literacy education. After the team has reviewed this list, the facilitator should 
create a list of relevant types of evidence for the group discussion. Team members will review the evidence that 
they have collected and compare it to the guiding questions listed for each part of the Scoring Guide as they com- 
plete the Implementation Majority Agreement Rating Form (see next section). 


Area 1: Development and use of student data profiles 


Part 1.1 A data collection process, beginning at student intake, is in place 
to collect and share data on student assessment and planning for proper 
program and instructional placement 


Possible sources of evidence 

e Student program progression plan. Program progression plans are typically developed by local adult literacy 
education programs and align with state policies and recommendations to identify criteria (such as course 
mastery, attendance, and assessment scores) that students must meet to advance to the next stage or level 
or to exit the program. 

e Program improvement plan with consideration for student outcomes. 

e Documentation of assessments and other criteria used to identify students’ literacy skills, including assess- 
ments such as GED, Test for Adult Basic Education, and Comprehensive Adult Student Assessment System; 
attendance; enrollment; prior academic experience; learning assessments; writing samples; student inter- 
views; and student program goals. 

¢ Documentation of student grades in academic courses and prior assessment scores. 

e Program schedule for administering assessments to monitor academic progress. 

e¢ Program evaluation/performance reporting and monitoring requirements. 


Part 1.2 Teachers/programs are able to track student progress through the 
use of valid and reliable assessments 


Possible sources of evidence 

e Student program progression plan. 

e Program improvement plan. 

e Instructional implementation timeline or program master schedule. 

e Instructional course schedule for adult literacy education teachers. 

e Student placement policy for adult literacy education and evidence that this policy is being followed. 
e Information provided to students on their progress and mastery of stated goals. 

¢ Documentation of criteria used to select assessments. 

e Documentation of assessments used to identify students’ literacy needs. 

¢ Documentation of timeline for regular (biweekly) student progress updates. 
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Area 2: Instructional methods and materials 


Part 2.1 A plan is in place to use criteria for selecting interventions and curricula 
that have been shown to have a positive effect on student achievement 


Possible sources of evidence 

e Review of current instructional materials for alignment with program standards and student needs. 
e ~=Review of criteria for selecting evidence-based literacy interventions and curricula. 

e Log or record of literacy interventions and curricula currently available at the school. 


Part 2.2 A plan is in place that reflects instructional practices shown through 
empirical evidence to increase student achievement 


Possible sources of evidence 

e Professional development plans for teachers, including instructional materials, an instructional schedule that 
maximizes instruction time, and instructional practices shown through empirical evidence to increase student 
achievement. 

e Instructional plans (lesson plans) for each course, including lesson plan templates. 

e Notes from interviews with administrators and teachers who implement courses. 

¢ Documentation of program and materials use. 

e Observations of course sessions documenting use of selected materials and aligned instructional practices. 

e Professional development attendance records and evaluations. 

e Progress monitoring tools and data. 

e Class syllabuses and timelines. 


Area 3: Instructional time 


Part 3.1 The school has established a schedule that will provide the 
appropriate frequency, consistency, and duration of instruction to meet 
adult learners’ needs 


Possible sources of evidence 

e Student progression plan. 

e Program and departmental master schedules (including literacy course schedules). 

e Interviews with students, community members, teachers, counselors, administrators, and staff to determine 
best schedules. 

e ~=Review of learning-management system data. 

e Review of attendance data. 


Area 4: Teacher preparation and training 


Part 4.1 A plan is in place to identify, hire, and retain faculty able to teach 
academic literacy skills effectively to adult learners 


Possible sources of evidence 

e Schedules for school faculty and staff. 

e Advertisements and marketing efforts for hiring teachers. 

¢ Documentation of hiring, onboarding, and scheduling work hours of teachers. 
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Lists of job descriptions, resumes, and interview rubrics for vetting teacher candidates. 
Records reflecting length of employment and full-time or part-time status of faculty and staff. 


Part 4.2 A plan is in place to provide professional development and ongoing 
support for teachers providing adult literacy education 


Possible sources of evidence 


Professional development schedule and training agenda. 

Professional development training materials. 

Plan for course observation. 

Fidelity checklist/rubric for implementation of professional development related to adult literacy education. 
Interviews with school faculty and staff responsible for organizing implementation of the program. 
Evidence of a faculty and staff orientation program. 

Evidence of a mentoring program for new faculty and for other teachers needing support. 


Area 5: Communication 


Part 5.1A plan is in place for communication and collaboration that will 
ensure successful startup and ongoing implementation of instructional 
programs 


Possible sources of evidence 


Documentation of faculty and staff roles and responsibilities. 

Marketing and public relations plans and minutes from student recruitment meetings. 
Interviews with administrators and leaders. 

Notes from planning meetings on adult literacy education. 

Memos from administration or leaders to the community. 

Documentation of communication with students and families. 

Results of student entrance interviews. 

Verification of student contact information, including phone numbers and email addresses. 
Minutes from advisory board meetings. 

Review of student retention data and census data. 

Schedule of school-sponsored community events. 

Contact information for wraparound service providers. 


Part 5.2 A plan is in place that enables teachers, administrators, counselors, 
and others involved in delivering adult literacy education to collaborate 
regularly on learners’ growth and program success and improvement 


Possible sources of evidence 


Interviews with adult literacy educators. 

Schedule of collaborative meetings between adult educator teams. 
Schedule of conferences with students. 

Organizational charts. 

Policy notices and reminders. 

Communication protocols with clear lines of communication. 

Memos to the community from administrators or other education leaders. 
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Area 6: Instructional environment and processes 


Part 6.1 A plan is in place to ensure a healthy and safe learning environment 
that is welcoming and conducive to learning 


Possible sources of evidence 


Documentation of program criteria for instructional environments. 

Documentation of a custodial/maintenance plan for instructional environments. 

Procedures for faculty and staff to report concerns about the instructional environment and for timely 
response to these concerns. 

Documentation of available instructional spaces to provide consistent instruction (inside and outside the 
classroom). 

Documentation of instructional materials (complete curricula) and supplies (manipulatives, pencils, paper, 
technology, erasers, pencil sharpeners) available and easily accessible for instructional use. 

Interviews with students and staff to confirm that the student experience is smooth and that students receive 
high-quality customer service. 

Documentation of processes for intake, assessment, and student orientation. 


Part 6.2 The learning environment is adaptive to the social, emotional, and 
cultural needs of adult learners 


Possible sources of evidence 


Documentation of training for faculty and staff on adult learners’ potential social, emotional, and cultural 
needs and on relevant services and supports for learners. 
Procedures for faculty and staff to identify and report concerns about adult learners’ social, emotional, or 
cultural needs and for timely response to these concerns. 
Interviews or questionnaires with learners and staff to confirm that the student experience is smooth and 
that social, emotional, and cultural needs are considered. 
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IMPLEMENTATION MAJORITY AGREEMENT 
RATING FORM 


Rating key: 

1= Not currently feasible 

2 = Feasible, but not implemented 
3 = In process of implementing 

4 = Fully implemented 

State: 
District/College/Institution: 


Program: 


The facilitator will complete this form by recording the results of group ratings and discussions from initial self- 
study results (see Planning and Implementation Scoring Guide), thoughts on priorities, and brainstorming ideas 
for next steps/activities for each area rated 2 or 3 (areas where development is most needed). 


Planning and Implementation Group rating 


Scoring Guide area ((olfgel(em allan] exeia) Priorities Ideas for next steps/activities 


Area1. Development and use of Part11: 1 2 3 4 
student data profiles 
Part1.2: 1 2 3 4 


Area 2. Instructional methods Part21: 12 3 4 
and materials 
Part2.2: 1 2 3 4 


Area 3. Instructional time Part31: 12 3 4 


Area 4. Teacher preparationand Part41: 1 2 3 4 


training 
Part4.2: 1 2 3 4 
Area 5. Communication Part51: 12 3 4 
Part5.2; 1 2 3 4 
Area 6. Instructional Part61: 12 3 4 


environment and processes 
Part6.2: 1 2 3 4 
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PLANNING NEXT STEPS FORM 


After completing the Implementation Majority Agreement Rating Form, the facilitator will begin completing this 
Planning Next Steps Form by leading a discussion on the priorities for action. Following that, the facilitator will 
lead a discussion on developing a detailed implementation plan for next steps and activities that are the most 
urgent and actionable. Finally, the facilitator will lead a discussion to identify potential challenges to the plan and 
resources available to help overcome those challenges. 


List the top three priority areas to improve implementation of evidence-based practices in adult literacy 
education. 


3: 
Priority Area 1 


What are the next steps and activities to address the listed priorities for this priority area? Consider timelines 
and the person or people responsible for determining the strategies or providing the resources. 


What are the general challenges the team anticipates? 


Outline a plan to address the challenges and tentatively identify the people responsible for each step of the 
plan to address this priority area. 
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Who will be responsible for monitoring progress as the plan is implemented? 


What will be the timeline for implementation? 


What budget and resources are needed to support successful implementation of the plan? 


Priority Area 2 


What are the next steps and activities to address the listed priorities for this priority area? Consider timelines 
and the person or people responsible for determining the strategies or providing the resources. 


What are the general challenges the team anticipates? 


Outline a plan to address the challenges and tentatively identify the people responsible for each step of the 
plan to address this priority area. 


Who will be responsible for monitoring progress as the plan is implemented? 
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What will be the timeline for implementation? 


What budget and resources are needed to support successful implementation of the plan? 


Priority Area 3 


What are the next steps and activities to address the listed priorities for this priority area? Consider timelines 
and the person or people responsible for determining the strategies or providing the resources. 


What are the general challenges the team anticipates? 


Outline a plan to address the challenges and tentatively identify the people responsible for each step of the 
plan to address this priority area. 


Who will be responsible for monitoring progress as the plan is implemented? 


What will be the timeline for implementation? 


What budget and resources are needed to support successful implementation of the plan? 
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SZ 


ACTION PLAN TEMPLATE 


Purpose: To create an Action Plan for an institution’s implementation of evidence-based practices in adult literacy education. 


Directions: Using this form as a template, develop an Action Plan identifying the action items your institution must accomplish to ensure that the priorities 


outlined in the Planning Next Steps Form are thoroughly addressed. 


Action steps 
What will 
NelUmolorg 


Step 1: 


Step 2: 


Resources 

e Resources 
available 

e Resources 
Tavatexe (tem Galar-lale-) A 
human, other) 

Coe Nolealialciva-nahyicy 
0] 6) oXe) am al=t-vel=xe| 


Timeline 
By when? (day/ 
mantel aluaVANc=x-1p) 


Responsibilities 
Who will do it? 


VEL yafey a) 


Potential barriers 

¢ What individuals . 
or entities Communications 
might resist? re) Ya} 

e How do we ¢ Who is involved? 
address this ¢ What methods? 
resistance? ¢ How often? 


raat=iaateye| 


How will you 
determine that 
you have reached 
NelU] a -XeY-] ia 

What are your 
measures? 


Budget 


Step 3: 


Step 4: 


Step 5: 


APPENDIX A. SUPPORT FOR PLANNING AND 
IMPLEMENTATION SCORING GUIDE AREAS 


This appendix describes key references that provide additional support for each of the Planning and Implementa- 
tion Scoring Guide areas. 


Area 1: Development and use of student data profiles 


Kruidenier, J. R., MacArthur, C. A., & Wrigley, H. S. (2010). Adult education literacy instruction: A review of the research. 
Washington, DC: National Institute for Literacy. https://eric.ed.gov/?id=ED521229, 


The authors note the importance of using assessments to determine each student’s reading ability, particularly 
to target instruction to meet individual student needs and monitoring progress. “Assessing a student’s reading 
ability is important in programs where reading instruction is individualized or where growth in reading is moni- 
tored” (p. 35). 


The authors also note the importance of using several data points to create student reading data profiles and 
develop specific instruction aligned with student reading goals. Student reading data profiles include results from 
all previous literacy assessments, enrollment information, and progress monitoring results. “An important finding 
in the Adult Education research suggests that assessing several components of reading in order to generate pro- 
files of students’ reading ability gives teachers much more instructionally relevant information than any test of a 
single component can” (p. 35). 


Lesgold, A. M., & Welch-Ross, M. (2012). Improving adult literacy: Options for practice and research. Washington, DC: Com- 
mittee on Learning Sciences: Foundations and Applications to Adolescent and Adult Literacy, National Research Council, 
National Academy of Sciences. https://eric.ed.gov/?id=ED536344. 


The authors note that it is important to have assessments that are aligned with common goals for learning and 
that provide information that is instructionally helpful for guiding student placement and understanding student 
progress and program outcomes. “Good systems of assessment to improve student learning consist of (a) diag- 
nostic assessment to inform instructors about skills the learner possesses and needs to develop; (b) formative 
assessment of skills being developed that need further improvement as instruction progresses; and (c) account- 
ability assessment to inform administrators, policy makers, funders, and the public of how well the program and 
systems that serve adult literacy learners are working. The assessments need to be aligned with common goals for 
learning. Assessments of literacy need to be suitable for adults, assess all the important dimensions of reading, 
writing, and language, and assess a range of print and digital functional literacy skills that society demands and 
values” (p. 4). In addition, the authors note the importance of identifying specific student needs. “It is vital to use 
reliable methods to diagnose learning and reading disabilities and to adjust instruction accordingly” (p. 78). 


The authors note the need for a systematic approach to using assessments in adult literacy education, with con- 
sideration for training and support. “A sound approach to assessment to support and monitor learning at the 
individual, program, and systems levels is systematic, with linkages among the various purposes of assessment 
and extensive professional training and supports needed to implement the assessments reliably” (p. 81). 


The authors note important considerations when selecting assessments in adult education settings. “For adult 
learners, Some assessment instruments are calibrated to important everyday literacy demands, and the scales 


from such instruments may be a far better indicator of adult literacy progress than grade equivalents” (p. 88). 
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The authors note the need for adult learners to self-assess and monitor their own progress toward their goals. 
“Learners reported several factors they perceived to help their progress: peer support, trusting the teacher, and 
explicit feedback, especially validation of their efforts and progress. Such findings indicate a need to develop 
various methods of assessment so that learners can continually assess themselves and each other to monitor 
progress toward learning goals” (p. 91). 


Area 2: Instructional methods and materials 


Kruidenier, J. R., MacArthur, C. A., & Wrigley, H. S. (2010). Adult education literacy instruction: A review of the research. 
Washington, DC: National Institute for Literacy. https://eric.ed.gov/?id=ED521229, 


The authors note the importance of selecting evidence-based instructional materials, implementing materials as 
designed, and employing evidence-based instructional delivery models for adult learners. Examples of evidence- 
based instructional methods and materials include multimedia technology (p. 24), direct instruction in compre- 
hension strategies (p. 104), and use of workplace-oriented materials (p. 101). 


Lesgold, A. M., & Welch-Ross, M. (2012). Improving adult literacy: Options for practice and research. Washington, DC: Com- 
mittee on Learning Sciences: Foundations and Applications to Adolescent and Adult Literacy, National Research Council, 
National Academy of Sciences. https://eric.ed.gov/?id=ED536344. 


The authors’ note: “Federal and state policy makers should move quickly to build on and expand the existing 
infrastructure of adult literacy education to support the use of instructional approaches, curricula, materials, 
tools, and assessments of learners consistent with (a) research on reading, writing, learning, language, and adult 
development; (b) research on the effectiveness of instructional approaches; and (c) knowledge of sound assess- 
ment practices” (p. 252). 


On research on instructional materials, the authors note: “Much of the available research on adult literacy 
describes the use of authentic texts gathered from actual contexts in which adults used these materials (e.g., 
a workplace, a restaurant) or ways of reading fiction and nonfiction and descriptions of how to match a learner 
with text and ‘debugging’ it to bring it into the ‘learners’ instructional zone’” (p. 94). 


The authors note that instructional methods should be adapted to student populations and needs. “Systematic 
implementation and evaluation of various approaches is required to identify the mix of strategies that will engage 
learners of different backgrounds for the large amounts of time required for instruction and practice to be effec- 
tive. The interventions should be developed with consideration of the factors that are likely to cause attrition and 
lack of perseverance. Programs can then select strategies that are most appropriate with an understanding of the 
specific situations of their students” (p. 253). 


Area 3: Instructional time 


Kruidenier, J. R., MacArthur, C. A., & Wrigley, H. S. (2010). Adult education literacy instruction: A review of the research. 
Washington, DC: National Institute for Literacy. https://eric.ed.gov/?id=ED521229, 


The authors note the importance of ensuring that schedules—course duration, length, and accessibility—meet 
the needs of adult education students. Examples of evidence-based recommendations for instructional time 
include focusing on intensity and duration (p. 102) and spending 70 percent or more of classroom time on literacy 
practice (p. 103). A stronger finding on intensity and duration is that “reading comprehension achievement may 
increase as a learner stays longer in a literacy program, although progress may be extremely variable over time” 
(p. 26). A weaker finding is that “spending a significant portion of classroom time practicing reading and writing, 
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including the occasional but direct or deliberate discussion of reading strategies, may increase learners’ meta- 
comprehension abilities” (p. 26). 


Lesgold, A. M., & Welch-Ross, M. (2012). Improving adult literacy: Options for practice and research. Washington, DC: Com- 
mittee on Learning Sciences: Foundations and Applications to Adolescent and Adult Literacy, National Research Council, 
National Academy of Sciences. https://eric.ed.gov/?id=ED536344. 


The amount of instructional time provided has been found to make a difference in adult education student 
reading outcomes. “On average, learners participated in adult education programs for less than 100 hours over 
the course of a program year, according to the Adult Education Program Survey. Only about one-third of adults 
made reading gains equivalent to a grade level during the program year. These findings are consistent with the 
levels of participation and progress reported in the few published studies of interventions designed to develop 
the literacy of adults with low-to-intermediate skills and other information gathered from individual researchers 
and practitioners working in the field. Reading is a complex skill, and research on the development of complex 
skills and expertise suggests that about 3,000 hours are required for mastery; 100 hours represent 3 percent 
of that amount, and so it is likely to be insufficient for learning for many adults, even if the goal is not expert 
mastery. Thus, one primary reason for limited progress may be that adults lack sufficient amounts of instruction 
and practice for improving skills” (p. 77). 


Scheduling and pacing of instructional time are found to be important for addressing concerns with student drop- 
outs. “Reasons reported for dropping out of adult education include family problems, the pace of instruction 
(either too fast or two slow), health issues, dislike of classwork, and inconvenient class location or schedule” 
(p. 77). 


Area 4: Teacher preparation and training 


Kruidenier, J. R., MacArthur, C. A., & Wrigley, H. S. (2010). Adult education literacy instruction: A review of the research. 
Washington, DC: National Institute for Literacy. https://eric.ed.gov/?id=ED521229, 


The authors note the importance of ensuring that adult literacy teachers are well trained and prepared for instruc- 
tion and that they are offered ongoing professional development and coaching. Examples of evidence-based 
recommendations for teacher preparation include hiring experienced, full-time teachers (p. 110) and providing 
teachers with classroom aides (p. 97). 


Lesgold, A. M., & Welch-Ross, M. (2012). Improving adult literacy: Options for practice and research. Washington, DC: Com- 
mittee on Learning Sciences: Foundations and Applications to Adolescent and Adult Literacy, National Research Council, 
National Academy of Sciences. https://eric.ed.gov/?id=ED536344. 


The authors note that adult literacy educators “are not consistently funded to participate in in-service profes- 
sional development, have access mostly to short-term training and conferences, and are hindered by systemic 
constraints from participating in professional development” (p. 79). 


Smith, C., & Gillespie, M. (2007). Research on professional development and teacher change: Implications for adult basic 
education. Review of Adult Learning and Literacy, 7(7), 205-244. 


The authors note that studies on professional development with adult basic education teachers found positive 
correlations, particularly when the professional development is of longer duration. “Professional development 
is more effective in changing teachers’ practice (at least as self-reported) when it is of longer duration.... Longer 
term professional development permits more time for teachers to learn about their own practice, especially if it 
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includes follow-up” (p. 216). The authors note that a study of change among adult basic education teachers “found 
a direct and positive correlation between the number of hours teachers participated in professional development 
activity and the amount and type of change related to the topic of the professional development they demon- 
strated in the following year” (p. 216). In addition, the authors note the importance of connecting teachers’ learn- 
ing with their work context, including knowledge application activities in professional development offerings, 
collaborating with teachers at the same location, and focusing on increasing subject-matter knowledge. 


Area 5: Communication 


Bryk, A. S., Gomez, L. M., & Grunow, A. (2011). Getting ideas into action: Building networked improvement communities in 
education. In Hallinan, M. T. (Ed.), Frontiers in Sociology of Education (pp. 127-162). Dordrecht, Netherlands: Springer. 
Retrieved March 2, 2020, from https://www.carnegiefoundation.org/wp-content/uploads/2014/09/bryk-gomez_building 
-nics-education.pdf. 


The authors note the importance of preparing communications and data collection processes that are sensitive to 
the immediate needs of students as they begin instruction in a new setting. “While summative lagging indicators 
are important, improvement research also needs data about specific program processes and student experiences 
as these occur in real time. This evidence is key for informing more micro-level activities linked to longer-term 
student success. For example, extant research suggests that the nature of students’ initial engagement with their 
community college during the first two or three weeks of enrollment is critical. Data about students’ academic 
behaviors and experiences during these critical weeks are key to understanding whether a pathway design is 
reducing early disengagement. Such data also may be used formatively to improve quick outreach efforts to stu- 
dents before they actually disengage” (p. 24). 


Hector-Mason, A., Narlock, J., Muhisani, H., & Bhatt, M. P. (2017). State strategies to facilitate adult learners’ transitions 
to postsecondary opportunities (REL 2017-223). Washington, DC: U.S. Department of Education, Institute of Education 
Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Midwest. 
https://eric.ed.gov/?id=ED571873. 


The authors note the importance of communication and community building in ensuring that students receive 
assistance transitioning to postsecondary training and careers. “Adult learners can face individual, institutional, 
and policy-related barriers to successful transitioning to postsecondary education. State directors of adult learner 
programs have discussed providing additional support services as a way to address these barriers. Respondents 
in Indiana, lowa, Michigan, and Minnesota discussed using such community-building approaches. However, only 
one state, Indiana, reported providing transportation assistance for adult learners who may have trouble attend- 
ing college and career transition programs” (p. 8). 


Area 6: Instructional environment and processes 


Kruidenier, J. R., MacArthur, C. A., & Wrigley, H. S. (2010). Adult education literacy instruction: A review of the research. 
Washington, DC: National Institute for Literacy. https://eric.ed.gov/?id=ED521229, 


The authors note the importance of ensuring that the instructional environment is conducive to learning for adult 
education students and that processes are in place to support student engagement and productivity. Examples of 
evidence-based recommendations for instructional environment include learner-centered classes (p. 100), com- 
munity building (p. 125), and differentiated teaching environments (p. 125). 


Lesgold, A. M., & Welch-Ross, M. (2012). Improving adult literacy: Options for practice and research. Washington, DC: Com- 
mittee on Learning Sciences: Foundations and Applications to Adolescent and Adult Literacy, National Research Council, 
National Academy of Sciences. https://eric.ed.gov/?id=ED536344. 
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The authors note the need for assessing learners and the instructional environment. “A valid, coherent, and com- 
prehensive system of assessment should be developed for diagnosis, planning instruction, and accountability. 
The system should comprehensively assess knowledge, skills, and valued psychological and functional outcomes. 
It should be aligned to produce different but linked forms of measurement for assessing learning at the learner, 
classroom, and program levels. The system should generate information that is appropriate and useful for the 
particular purposes and audiences: learners, instructors, program administrators, or policy makers in local, state, 
and federal governments. Effective assessment tools would address all of the components of literacy and map 
onto the primary valued learning outcomes for adult remedial and basic education” (p. 258). 


The authors note specific features of learning environments that may be most beneficial for adult learners. “A 
number of studies suggest that programs that create supportive learning environments that take into account 
adults’ prior life experiences and offer opportunities for self-directed learning seem to set the stage for successful 
learning experiences” (p. 400). “In adult learning settings, successful programs are ones in which students are 
supported individually, are given enough time to work on computers (including for personal purposes), and allow 
collaboration between students and between teachers and students” (p. 400). 


The authors also found that certain types of learning environments may have inherent challenges. “However, 
learning environments that include top down administration, particularly involving staff members who believe 
that technology is neutral and who are not adequately trained, can lead to resistance, dropping out, or other 
problems with participant attitudes toward learning. Furthermore, the presence of technologies alone in these 
programs cannot overcome the social and cultural inequities that affect adults’ beliefs and attitudes toward 
lifelong learning and technology, which can also affect teachers’ integration of them into adult basic education. 
After years of absence from formal learning situations or having negative earlier schooling experiences, adult stu- 
dents can be intimidated by overly structured, test-centered programs. Many times these programs, full of young 
people, presume basic computer literacy or English proficiency, and they do not take into account how adults 
who have not been involved with ICT (information and communication technologies) use can be intimidated and 
anxious about adopting these new roles in unfamiliar educational settings” (p. 400). 
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